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Abstract 


In  1993,  the  Carnegie  Mellon®  Software  Engineering  Institute  (SEI)  developed  a  taxonomy- 
based  method  for  facilitating  the  systematic  and  repeatable  identification  of  risks  associated 
with  the  development  of  a  software-dependent  project.  Since  then,  this  method  has  also  been 
used  in  the  Software  Risk  Evaluation  process  to  identify  risks  associated  with  the  development 
of  software-intensive  systems. 

Recently,  organizations  that  employ  software-intensive  systems  have  requested  that  the  SEI 
help  identify  a  baseline  set  of  risks  associated  with  missions  performed  at  operational  sites 
(e.g.,  satellite  ground  stations,  military  units,  customer  service  units).  While  the  concepts 
embodied  in  the  software-based  taxonomy  apply  in  this  context,  the  taxonomy  presented  in 
this  report  has  been  constructed  to  better  suit  an  operational  environment. 

This  report  presents  a  taxonomy-based  method  for  identifying  and  classifying  risks  to  opera¬ 
tional  aspects  of  an  enterprise.  It  defines  the  key  sources  of  risk  associated  with  the  mission, 
work  processes,  and  constraints  of  an  operational  organization  and  establishes  a  structure  for 
representing  operational  risks  by  grouping  them  into  distinct  classes,  elements,  and  attributes. 
In  addition,  the  appendix  of  this  report  contains  a  short  taxonomy-based  questionnaire  that  can 
be  used  by  personnel  at  operational  sites  to  identify  and  categorize  risks. 
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1  Introduction 


The  identification  of  risks  is  a  major  function  of  the  SEI’s  Continuous  Risk  Management  pro¬ 
cess,  along  with  analysis,  planning,  tracking,  controlling,  and  communicating  risks.  Risk  iden¬ 
tification  is  a  crucial  element  of  successful  risk  management  in  both  developmental  and 
operational  undertakings. 

As  with  the  taxonomy -based  risk  identification  method  for  software  development  projects 
published  by  the  SEI  in  1993,  the  perspective  taken  in  this  report  is  that  there  are  risks  inherent 
in  missions  performed  at  operational  sites.  Furthermore,  taking  risks  is  essential  to  progress 
and  failure  is  often  a  key  part  of  learning.  Existing  approaches  to  risk  management  tend  to  be 
ad  hoc,  undocumented,  incomplete,  and  dependent  on  the  experience  and  risk  orientation  of 
key  personnel  at  operational  sites.  However,  project  risks  are  usually  known  by  personnel 
(though  they  might  not  use  the  term  risk  to  describe  them)  and  as  a  consequence  can  be  sur¬ 
faced  and  managed  [Carr  93]. 

The  taxonomy  of  operational  risks  provides  a  structure  for  classifying  risks  to  operational 
aspects  of  an  enterprise.  It  defines  the  key  sources  of  risk  associated  with  the  mission,  work 
processes,  and  constraints  of  an  operational  organization  and  establishes  a  structure  for  repre¬ 
senting  operational  risks  by  grouping  them  into  distinct  classes,  elements,  and  attributes.  The 
short  taxonomy-based  questionnaire  included  in  this  report  can  be  used  by  personnel  at  opera¬ 
tional  sites  to  identify  and  categorize  of  risks. 

1.1  Audience  and  Context 

The  taxonomy  of  operational  risks  is  primarily  intended  for  use  by  personnel  at  operational 
sites  such  as  satellite  ground  stations,  military  units,  and  customer  service  units  that  focus  on 
completing  operational  missions.  An  operational  mission  comprises  tasks  that  involve  the 
practical  application  of  principles  or  processes. 

1.2  Document  Organization 

The  taxonomy  of  operational  risks  presented  in  this  document  is  organized  as  follows 

•  Section  1  —  Introduction 

•  Section  2  —  Class,  Element,  and  Attribute  Definitions 

•  Section  3  —  Conclusion 

•  Appendix  —  A  Short  Taxonomy-Based  Questionnaire  for  Identifying  Operational  Risks 
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The  first  section  provides  a  brief  overview  of  the  perspective  taken  when  developing  the  tax¬ 
onomy  of  operational  risks  and  describes  the  audience  for  and  organization  of  the  report.  Sec¬ 
tion  2  contains  the  taxonomy  of  operational  risks  grouped  according  to  class,  element,  and 
attribute  with  definitions  provided  for  each  entry.  The  conclusion  contains  a  summary  of  the 
report  contents  and  directions  for  future  work.  Finally,  the  appendix  contains  a  short  taxon¬ 
omy-based  questionnaire  that  can  be  used  by  personnel  at  operational  sites  to  facilitate  the 
identification  and  categorization  of  risks. 
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2  Class,  Element,  and  Attribute  Definitions 


This  section  describes  the  hierarchy  of  classes,  elements,  and  attributes  that  comprise  the  tax¬ 
onomy  of  operational  risks.  As  shown  in  Table  1,  the  taxonomy  is  organized  into  three  main 
classes : 

•  Mission  —  addresses  characteristics  of  the  operational  mission  itself,  mechanisms  to 
request  products  or  services,  and  the  outputs  of  the  operational  mission 

•  Work  Processes  —  addresses  aspects  of  the  choices  the  operational  organization  makes 
when  deciding  how  to  execute  the  mission 

•  Constraints  —  identifies  external  influences  that  affect  the  operational  mission 

Each  of  these  classes  is  divided  into  its  constituent  parts,  called  elements.  Each  element  is 
associated  with  a  set  of  attributes,  or  characteristics,  traits,  qualities,  or  properties,  that  are 
used  to  describe  the  element.  Individual  classes,  elements,  and  attributes  are  described  in  detail 
in  the  remainder  of  this  section. 
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Table  1:  The  Taxonomy  of  Operational  Risks 


A.  Mission 

1.  Tasking,  Orders,  and 
Plans 

a.  Stability 

b.  Completeness 

c.  Clarity 

d.  Validity 

e.  Feasibility 

f.  Precedent 

g.  Timeliness 

2.  Mission  Execution 

a.  Efficiency 

b.  Effectiveness 

c.  Complexity 

d.  Timeliness 

e.  Safety 

3.  Product  or  Service 

a.  Usability 

b.  Effectiveness 

c.  Timeliness 

d.  Accuracy 

e.  Correctness 

4.  Operational  Systems 

a.  Throughput 

b.  Suitability 

c.  Usability 

d.  Familiarity 

e.  Reliability 

f.  Security 

g.  Inventory 

h.  Installations 

i.  System  Support 


B.  Work  Processes 

1.  Operational  Processes 

a.  Formality 

b.  Suitability 

c.  Process  Control 

d.  Familiarity 

e.  Product  Control 

2.  Maintenance  Processes 

a.  Formality 

b.  Suitability 

c.  Process  Control 

d.  Familiarity 

e.  Service  Quality 

3.  Management  Processes 

a.  Planning 

b.  Organization 

c.  Management  Experience 

d.  Program  Interfaces 

4.  Management  Methods 

a.  Monitoring 

b.  Personnel  Management 

c.  Quality  Assurance 

d.  Configuration 
Management 

5.  Work  Environment 

a.  Quality  Attitude 

b.  Cooperation 

c.  Communication 

d.  Morale 


C.  Constraints 

1.  Resources 

a.  Schedule 

b.  Staff 

c.  Budget 

d.  Facilities 

e.  Tools 

2.  Policies 

a.  Laws  and  Regulations 

b.  Restrictions 

c.  Contractual  Constraints 

3.  Interfaces 

a.  Customer/User 
Community 

b.  Associate  Agencies 

c.  Contractors 

d.  Senior  Leadership 

e.  Vendors 

f.  Politics 
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A.  Mission 


In  an  operational  environment,  a  mission  is  considered  to  be  the  primary  reason  for  the  exist¬ 
ence  of  the  operational  organization.  The  mission  consists  of  a  set  of  defined  tasks  that  pro¬ 
duce  a  product  or  service  for  a  customer.  The  mission  could  be  defense  intelligence  operations, 
banking,  retail  sales,  manufacturing,  or  a  variety  of  other  missions,  including  those  performed 
by  civil  agencies. 

The  elements  of  the  Mission  class  of  operational  risks  cover  traditional  aspects  of  the  mission, 
including  planning,  execution,  and  the  products  and  services  provided.  Mission  elements 
include  attributes  of  the  operational  systems  and  the  organizations  that  operate  those  systems. 

1.  Tasking,  Orders,  and  Plans 

The  Tasking,  Orders,  and  Plans  element  contains  attributes  that  are  used  to  characterize 
aspects  of  the  information  contained  in  the  tasks,  orders,  and  plans  of  an  operational  organiza¬ 
tion.  These  attributes  also  describe  the  ability  of  an  operational  system  and  the  organization 
that  operates  it  to  respond  to  requests.  The  following  attributes  characterize  the  Tasking, 
Orders,  and  Plans  element. 

a.  Stability 

The  Stability  attribute  refers  to  the  frequency  with  which  tasks,  orders,  or  plans  change 
and  the  effect  this  has  on  the  operational  organization.  It  can  also  refer  to  the  organiza¬ 
tions  that  submit  tasks  or  orders  to  an  organization  for  execution.  This  attribute  also 
addresses  the  flexibility  of  the  operational  entity  in  responding  to  changing  tasks, 
orders,  and  plans  and  to  handling  multiple  sources  of  tasks,  orders,  and  plans. 

b.  Completeness 

Complete  information  in  tasks,  orders,  or  plans  such  as  time  of  task  initiation,  the  period 
during  which  the  task  is  to  be  performed,  the  end  time  of  the  task,  the  outputs  of  the 
task,  or  the  recipient  of  the  results  are  critical  to  ensuring  the  successful  execution  of  the 
task,  order,  or  plan. 

c.  Clarity 

Clarity  of  a  task,  order,  or  plan  is  an  attribute  that  enables  the  operational  organization 
to  clearly  understand  what  the  customer  needs  or  wants.  A  clearly  stated  request  or  plan 
requires  little  or  no  interpretation  by  the  operational  organization;  interpretation  by  the 
operational  organization  introduces  the  risk  of  producing  an  inadequate  or  undesired 
result. 
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d.  Validity 

The  validity  of  a  task,  order,  or  plan  refers  to  its  appropriateness  with  respect  to  the  pur¬ 
pose,  goals,  and  capabilities  of  the  operational  organization’s  systems.  Validity  of  a 
task,  order,  or  plan  can  also  refer  to  its  internal  consistency  or  other  parameters  such  as 
requesting  authority. 

e.  Feasibility 

The  Feasibility  attribute  refers  to  the  operational  organization’s  ability  to  meet  the 
requests  of  the  customer.  Feasibility  considers  the  actual  capability  of  the  organization’s 
systems  to  provide  the  necessary  product  or  service  and  also  the  capability  to  provide 
products  or  services  to  multiple  customers.  For  example,  system  capacity  can  pose  a 
risk  or  constraint.  In  addition,  geographic  or  seasonal  constraints  can  affect  feasibility. 

Also  included  in  this  attribute  is  the  ability  to  identify  a  viable  quantification  method  for 
measuring  the  results  to  determine  if  the  system  has  satisfied  the  request  for  products  or 
services. 

f.  Precedent 

The  Precedent  attribute  addresses  the  ability  of  an  operational  organization  to  perform 
tasks  that  it  has  not  performed  previously  or  may  not  have  been  designed  to  perform.  In 
addition,  it  is  possible  that  the  organization’s  operational  systems  may  not  have  been 
intended  for  a  specific  type  of  task  and  there  may  be  political  issues  or  risks  with  using 
a  system  in  an  unintended  way. 

In  some  cases,  where  precedent  itself  may  not  pose  significant  risk,  there  may  be  risk  in 
reconfiguring  the  system  to  perform  standard  operations. 

g.  Timeliness 

The  Timeliness  attribute  refers  to  the  operational  organization  receiving  the  task,  orders, 
or  requests  for  services  to  allow  the  scheduling  and  performance  of  the  task  in  a  time 
frame  that  meets  the  requestor’s  needs.  Processes  used  to  receive  tasks,  prioritize  them, 
and  prepare  for  execution  may  pose  risks  to  timely  planning  and  execution. 

Lack  of  customer  understanding  or  knowledge  regarding  required  lead  times  can  pose 
risks  to  customer  satisfaction. 

2.  Mission  Execution 

Attributes  of  the  Mission  Execution  element  are  used  to  characterize  the  ability  of  the  opera¬ 
tional  organization  to  perform  tasks  in  an  effective  and  timely  way.  The  complexity  of  prepar¬ 
ing  the  system  for  customer  tasks  is  also  an  attribute  of  the  Mission  Execution  element. 


6 


CMU/SEI-2005-TR-036 


a.  Efficiency 

The  Efficiency  attribute  depends  on  systems  and  personnel  being  prepared  to  accept 
new  or  updated  tasks  or  orders  and  execute  them  in  time  to  meet  customer  needs.  Exe¬ 
cution  efficiency  refers  to  the  ability  to  prioritize  tasks  and  orders  according  to  multiple 
parameters  such  as  schedule,  customer  priority,  and  geographic  considerations. 

b.  Effectiveness 

The  Effectiveness  attribute  describes  the  organization’s  ability  to  meet  customers’  needs 
according  to  their  tasks  and  orders.  Included  in  effectiveness  is  the  ability  to  evaluate 
effectiveness  using  measures  that  have  been  defined  for  assessing  effectiveness  for  each 
task,  order,  or  plan. 

c.  Complexity 

The  Complexity  attribute  includes  several  aspects  of  mission  execution.  For  example,  a 
task  that  requires  difficult  or  complex  communication  may  be  subject  to  more  operator 
errors  than  one  for  which  communication  is  easier  and  standardized.  Complexity  in 
using  or  configuring  an  operational  system  can  require  additional  time,  which  could 
result  in  lack  of  timely  responses  to  customer  requests.  Complex  interfaces  among 
diverse  organizations,  customers,  or  system  components  can  also  reduce  timeliness  and 
accuracy. 

d.  Timeliness 

In  the  context  of  the  Mission  Execution  element,  the  Timeliness  attribute  refers  to  the 
ability  of  operational  systems  and  personnel  to  execute  the  requested  task  (or  to  process 
the  order)  in  a  time  frame  that  meets  the  customer’s  needs.  This  attribute  depends  on  the 
work  processes  of  the  operational  organization.  Work  processes  include  understanding 
the  request,  scheduling  the  task,  and  communicating  commands  or  instructions  to  the 
operational  systems  or  personnel  in  time  to  perform  the  task  to  obtain  the  desired  result. 
Timeliness  also  depends  on  the  ability  of  the  operational  systems  to  be  reconfigured  in  a 
timely  fashion,  if  necessary. 

e.  Safety 

The  Safety  attribute  relates  to  potential  hazards  from  performing  the  day-to-day  mission 
and  the  operational  organization’s  ability  to  identify  and  manage  those  hazards.  Risks  in 
the  safety  area  may  be  related  to  performing  unprecedented  tasks  or  operating  in  a  hos¬ 
tile  environment.  Risks  can  arise  from  the  operational  organization’s  inability  to  value 
an  individual  voice  when  raising  and  dealing  with  safety  issues  or  concerns  efficiently 
and  effectively. 
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3.  Product  or  Service 

The  Product  or  Service  element  of  the  Mission  class  contains  attributes  that  are  used  to 
describe  the  products  that  the  operational  organization  produces  or  the  services  that  it  performs 
for  the  customer.  The  following  attributes  characterize  the  Product  or  Service  element  and 
reflect  the  level  of  customer  satisfaction  attained  with  the  product  or  service. 

a.  Usability 

The  Usability  attribute  of  a  product  or  service  is  used  to  characterize  human-system 
interaction  with  regard  to  ease  with  which  customers  can  use  the  delivered  product  or 
service  to  meet  their  goals.  In  this  context,  usability  can  depend  on  correct  configuration 
for  use  in  customer  environments,  system  accuracy,  or  timely  delivery.  Occasionally, 
the  product  or  service  provided  by  the  operational  organization  is  what  the  customer 
requested,  but  the  requirements  that  the  customer  submitted  yielded  a  product  or  service 
that  did  not  ultimately  meet  their  needs.  In  this  situation,  usability  risks  can  be  related  to 
the  Clarity  attribute  of  the  Tasking,  Orders,  and  Plans  mission. 

b.  Effectiveness 

The  Effectiveness  attribute  refers  to  the  ability  of  a  product  or  service  to  satisfy  needs  of 
the  customer.  Product  effectiveness  parameters  can  include  timeliness,  accuracy,  and 
correctness,  for  example.  Service  effectiveness  parameters  may  include  timeliness, 
completeness,  ease  of  interaction,  for  example.  Effectiveness  is  measured  using  criteria 
established,  in  advance,  by  the  customer  in  cooperation  with  the  operational  personnel. 
Effectiveness  may  be  dependent  on  the  ability  to  establish  and  manage  realistic  cus¬ 
tomer  expectations. 

c.  Timeliness 

The  Timeliness  attribute  refers  to  delivery  of  a  product  to  the  customer  during  the  time 
frame  requested.  Timeliness  depends  on  the  operational  organization’s  ability  to  prepare 
and  deliver  results  to  the  user  within  the  time  agreed  to  by  both  parties.  Product  delivery 
can  also  depend  on  external  communications  or  other  external  organizations  or  systems 
that  could  present  a  risk  to  timely  delivery. 

d.  Accuracy 

The  Accuracy  attribute  refers  to  the  degree  to  which  the  output  reflects  the  actual  condi¬ 
tions  or  real-world  data.  For  example,  the  output  of  the  Global  Positioning  System  pro¬ 
vides  results  that  are  not  perfect,  but  have  a  high  degree  (within  100  meters)  of 
accuracy. 


8 


CMU/SEI-2005-TR-036 


e.  Correctness 

The  Correctness  attribute  describes  the  degree  to  which  the  product  represents  the  prod¬ 
uct’s  design.  For  example,  when  a  product  depends  on  processing  algorithms,  correct¬ 
ness  refers  to  the  degree  to  which  the  algorithm  produces  the  expected  results  across  the 
range  of  operational  parameters. 

4.  Operational  Systems 

The  Operational  Systems  element  of  the  Mission  class  contains  attributes  that  are  used  to  char¬ 
acterize  the  operational  systems'  ability  to  perform  satisfactorily  and  the  features  that  affect  an 
operator’s  ability  to  interact  with  systems  to  produce  products  or  perform  services  for  the  cus¬ 
tomer.  The  following  attributes  characterize  the  Operations  Systems  element. 

a.  Throughput 

The  Throughput  attribute  refers  to  operational  systems’  ability  to  satisfy  the  anticipated, 
combined  needs  of  its  customers.  Throughput  risks  can  be  related  to  the  designed  capac¬ 
ity  of  the  system  as  determined  by  the  expected  number  of  customers  and  the  output 
needs  of  each.  Throughput  risks  can  also  be  related  to  specific  high-volume  periods  or 
events. 

b.  Suitability 

The  Suitability  attribute  describes  specific  features  that  enable  operational  systems  to 
meet  customer  needs  in  conjunction  with  the  planning  and  execution  needs  of  the  oper¬ 
ational  organization.  Issues  with  suitability  can  include  inappropriate  use  or  modifica¬ 
tion  of  an  existing  system  to  meet  new  mission  requirements  or  needs.  Also,  over- 
design  of  a  system  or  inappropriate  use  of  technology  (old  or  new)  can  pose  risk. 

c.  Usability 

The  Usability  attribute  of  an  operational  system  is  used  to  characterize  human-system 
interaction  with  regard  to  the  ease  with  which  operators  can  achieve  tasks  in  a  particular 
environment  or  by  using  a  specific  product.  For  example,  the  use  of  standard  icons  or 
positioning  of  items  on  interfaces  can  make  using  the  system  more  “intuitive,”  enhanc¬ 
ing  usability.  Providing  operators  with  direct  access  to  the  needed  components  of  a  sys¬ 
tem  can  also  enhance  usability.  The  Usability  attribute  can  also  be  used  to  describe  the 
physical  location  of  system  components  accessed  by  system  operators. 

d.  Familiarity 

The  Familiarity  attribute  describes  personnel’s  knowledge  of,  experience  in,  and  com¬ 
fort  with  the  operational  systems  used  to  perform  the  mission.  Familiarity  reflects  a 
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combination  of  formal  or  on-the-job  training,  mentoring,  and  experience.  Risks  related 
to  the  Familiarity  attribute  can  include  low  productivity  and  low-quality  results. 

e.  Reliability 

The  reliability  of  operational  systems  includes  availability  to  process  tasks  and  orders  in 
the  manner  described  in  the  operational  concept  or  in  the  system  requirements  and  the 
ability  to  achieve  repeatable  results.  Availability  can  be  affected  by  hardware  perfor¬ 
mance  or  by  system  complexities  that  affect  reconfiguration  (hardware  and  software) 
following  a  system  failure.  Repeatable  results  refers  to  the  operational  organization’s 
ability  to  produce  the  same  results  for  products  or  services  when  given  the  same  tasks  or 
orders. 

f.  Security 

The  Security  attribute  characterizes  data,  system,  and  inter-system  security  aspects  that 
can  introduce  risk  to  the  operation.  Data  security  refers  the  integrity  of  individual  data 
elements  within  the  operation.  System  security  refers  to  the  ability  to  restrict  access  to 
components  within  the  system.  Inter-system  security  refers  to  the  ability  to  ensure  the 
integrity  of  external  interfaces  and  restrict  access  by  external  systems. 

g.  Inventory 

The  Inventory  attribute  describes  aspects  of  managing  and  using  raw  materials  to  pro¬ 
duce  products  or  services  during  mission  execution.  Risks  in  this  area  relate  to  the 
unavailability  of  resources  when  required  or  the  costs  of  having  to  store  large  amounts 
of  raw  materials  waiting  to  be  processed. 

h.  Installations 

The  Installations  attribute  describes  aspects  of  operational  acceptance  and  the  integra¬ 
tion  of  new  capabilities  into  the  operational  environment.  Risks  in  this  area  are  associ¬ 
ated  with  depletion  of  capability  or  throughput,  increases  in  operational  expenditures,  or 
less  than  optimal  inventory  levels  resulting  from  system  upgrades. 

i.  System  Support 

The  System  Support  attribute  describes  the  ability  of  facilities  and  other  personnel  to 
provide  adequate  system  support  to  meet  availability,  reliability,  capacity,  and  other 
operational  requirements.  System  support  can  include  repair/replace  activities  and  facil¬ 
ities  maintenance.  It  can  also  include  the  collection  and  analysis  of  system  performance 
data  to  identify  areas  for  improvement. 
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B.  Work  Processes 


The  Work  Processes  class  contains  elements  and  attributes  that  are  used  to  describe  the  pro¬ 
cesses  through  which  the  operational  organization  fulfills  its  mission.  The  element  and 
attributes  in  this  class  can  also  be  used  to  characterize  the  management  processes,  methods, 
and  environment  in  which  the  work  takes  place. 

1 .  Operational  Processes 

The  attributes  contained  in  the  Operational  Processes  element  are  used  to  describe  the 
sequence  of  steps,  inputs,  outputs,  actions,  verification,  and  monitoring  activities  that  the  oper¬ 
ational  organization  follows  to  provide  its  products  or  services. 

Risks  in  this  category  surface  as  a  result  of  operational  processes  that  are  inadequately 
planned,  defined,  and  documented,  those  that  are  not  suited  to  the  activities  necessary  to 
accomplish  the  operational  mission,  those  that  are  poorly  communicated  to  the  staff,  or  those 
that  are  not  enforced.  The  following  attributes  characterize  the  Operational  Processes  element. 

a.  Formality 

The  Formality  attribute  of  the  operational  processes  element  describes  the  degree  to 
which  operational  processes  are  defined,  documented,  and  communicated.  Lack  of  suf¬ 
ficient  formality  can  produce  inconsistent  results.  A  lack  of  sensitivity  with  regard  to 
organizational  culture  can  hamper  the  adoption  of  formal  processes.  For  examples,  mil¬ 
itary  processes  can  have  a  different  level  of  formality  than  warehousing  operations. 

b.  Suitability 

The  Suitability  attribute  addresses  how  adequately  the  scope  of  the  operation  and  types 
of  activities  that  comprise  the  operation  are  supported.  Processes  that  are  incomplete  or 
outdated  can  represent  risks  to  completing  the  operational  mission. 

c.  Process  Control 

The  Process  Control  attribute  not  only  ensures  the  consistent  use  of  operational  pro¬ 
cesses  but  also  consistent  measurement  and  improvement  of  processes  based  on  obser¬ 
vations  of  the  quality  of  the  results  and  productivity  goals. 

d.  Familiarity 

The  Familiarity  attribute  describes  how  familiar  personnel  are  with  the  operational  pro¬ 
cesses,  It  covers  their  knowledge  of,  experience  in,  and  comfort  with  the  prescribed  pro¬ 
cesses.  It  reflects  a  combination  of  formal  or  on-the-job  training,  mentoring,  and 
experience.  Risks  in  this  area  can  include  low  productivity  and  low-quality  results. 
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e.  Product  Control 

Product  control  refers  to  the  monitoring  of  the  product  or  service  quality  or  other 
aspects  of  the  final  product  or  service  at  specific  points  in  the  operational  processes  to 
ensure  the  integrity  of  the  final  result.  Products  or  services  that  do  not  meet  quality  stan¬ 
dards  can  result  in  customer  dissatisfaction. 

2.  Maintenance  Processes 

Maintenance  Processes  are  processes  that  ensure  that  the  operational  infrastructure  (including 
equipment,  software,  utilities,  and  communications)  can  effectively  support  the  operational 
mission.  Some  risks  identified  in  this  category  can  be  associated  with  outsourcing  some  or  all 
of  the  maintenance  processes.  The  following  attributes  characterize  the  Maintenance  Pro¬ 
cesses  element. 

a.  Formality 

Formality  of  maintenance  processes  addresses  the  degree  to  which  maintenance  pro¬ 
cesses  are  defined,  documented,  and  communicated.  Lack  of  sufficient  processes  and 
procedures  to  guide  individuals  who  perform  maintenance  tasks  can  result  in  inadequate 
performance.  Too  little  formality  can  put  maintenance  results  at  risk  or  produce  incon¬ 
sistent  results.  Too  formal  a  process  can  cause  some  maintenance  needs  to  be  ignored. 

b.  Suitability 

The  Suitability  attribute  with  regard  to  maintenance  processes  describes  the  scope  of 
maintenance,  consistency  with  operational  infrastructure,  and  compatibility  with  rou¬ 
tine  mission  needs.  Suitability  includes  adequate  maintenance  plans  to  address  the  com¬ 
plete  scope  of  the  operational  infrastructure  and  keeping  maintenance  processes  current 
with  infrastructure  updates.  It  also  includes  planning  maintenance  activities  so  that  the 
maintenance  schedule  is  compatible  with  routine  mission  needs. 

c.  Process  Control 

The  Process  Control  attribute  refers  not  only  to  ensuring  the  use  of  the  maintenance  pro¬ 
cesses,  but  also  to  the  measurement  and  improvement  of  processes  based  on  the  quality 
of  the  results  and  availability  of  infrastructure  components.  Process  control  includes 
providing  mechanisms  for  monitoring  process  implementation  and  measuring  process 
results  for  improvement  purposes. 

d.  Familiarity 

Familiarity  with  maintenance  processes  characterizes  knowledge  of,  experience  in,  and 
comfort  with  the  prescribed  processes.  It  reflects  a  combination  of  formal  or  on-the-job 
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training,  mentoring,  and  experience.  It  includes  planning  to  reduce  the  impact  of  new 
and  unfamiliar  processes. 

e.  Service  Quality 

The  Service  Quality  attribute  refers  to  the  monitoring  of  the  maintenance  service  quality 
or  other  aspects  of  maintenance  service  at  specific  points  in  the  maintenance  processes 
to  ensure  the  integrity  of  the  final  results.  It  includes  mechanisms  to  measure  opera¬ 
tional  system  availability,  time  to  repair  or  update,  and  other  measures  to  control  the 
quality  of  maintenance  activities. 

3.  Management  Processes 

The  attributes  contained  in  the  Management  Processes  element  are  used  to  characterize  risks 
associated  with  planning,  monitoring,  and  controlling  the  operational  budget  and  schedule; 
with  managing  operational  personnel;  and  with  handling  external  organizations  including  cus¬ 
tomers,  contractors,  and  other  agencies.  The  following  attributes  characterize  the  Management 
Processes  element. 

a.  Planning 

The  Planning  attribute  characterizes  the  risks  associated  with  developing  well-defined 
operational  plans,  formulated  with  the  input  and  consent  of  those  affected  by  the  plans, 
that  respond  to  contingencies  and  overarching  goals  of  the  mission.  Also  included  are 
the  tasks  of  managing  the  project  according  to  the  plans  and  formally  modifying  the 
plans  when  changes  are  necessary.  The  mismatch  of  plans  to  operational  needs  (for  flex¬ 
ibility  or  rapid  re-orientation)  can  be  a  source  of  risk. 

b.  Organization 

The  Organization  attribute  describes  the  effectiveness  of  the  organizational  structure  of 
the  operational  organization  as  it  relates  to  carrying  out  a  specific  operational  mission.  It 
includes  clear  definitions  of  roles  and  responsibilities  and  assurance  that  operational 
personnel  understand  these  roles  and  the  decision-making  hierarchy. 

c.  Management  Experience 

The  Management  Experience  attribute  characterizes  the  experience  of  all  levels  of  man¬ 
agers  with  respect  to  their  management  ability,  domain  familiarity,  ability  to  deal  with 
scale  and  complexity  issues,  familiarity  with  operational  processes,  and  hands-on  oper¬ 
ational  experience. 
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d.  Program  Interfaces 

The  Program  Interfaces  attribute  describes  the  interactions  of  managers  at  all  levels 
with  the  operational  personnel  at  all  levels  and  with  external  personnel  such  as  peer 
managers,  senior  management,  and  customers. 

4.  Management  Methods 

The  Management  Methods  element  contains  attributes  that  are  used  to  characterize  an  opera¬ 
tional  organization’s  methods  for  managing  the  mission  and  operational  personnel.  These 
methods  include  quality  assurance,  configuration  management,  staff  development  with  respect 
to  operational  needs,  and  level  of  communication  about  operational  status  and  needs.  The  fol¬ 
lowing  attributes  characterize  the  Management  Methods  element. 

a.  Monitoring 

Monitoring  includes  activities  for  obtaining  and  acting  upon  status  reports,  allocating 
status  information  to  the  appropriate  operational  personnel,  and  maintaining  and  using 
operational  metrics.  Lack  of  monitoring  can  result  in  an  uncontrolled  process  and  inap¬ 
propriate  or  ineffective  actions  of  operational  personnel  due  to  lack  of  feedback. 

b.  Personnel  Management 

Personnel  management  describes  how  operational  personnel  are  selected  and  trained. 
The  operational  organization  should  ensure  that  they  take  part  in  planning  and  customer 
interaction  for  their  areas  of  responsibility,  work  according  to  plan,  and  receive  the  help 
they  need  or  ask  for  to  carry  out  their  responsibilities.  This  attribute  includes  planning 
growth  and  development  opportunities  for  operational  personnel. 

c.  Quality  Assurance 

The  Quality  Assurance  attribute  describes  the  procedures  instituted  to  ensure  that  both 
the  operational  processes  and  standards  are  implemented  properly  for  all  operational 
activities  and  that  the  quality  assurance  function  is  adequately  staffed  to  perform  its 
duties.  It  includes  identifying  and  using  mechanisms  to  collect  process  implementation 
data  and  measures  and  a  role  definition  for  quality  assurance  to  include  process  imple¬ 
mentation  assurance. 

d.  Configuration  Management 

The  Configuration  Management  (CM)  attribute  describes  both  staffing  and  tools  for  the 
operational  CM  function  and  the  complexity  of  the  required  CM  process  with  respect  to 
factors  like  having  multiple  operational  sites.  It  involves  managing  the  configuration  of 
both  hardware  and  software  systems  that  support  the  operation  in  a  manner  that  pro- 
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vides  the  end  user  with  consistent,  compatible  products  and  services.  Poor  communica¬ 
tion  with  system  development  organizations  can  increase  risk  in  this  area. 

5.  Work  Environment 

The  Work  Environment  element  contains  attributes  that  are  used  to  characterize  the  subjective 
aspects  of  the  operational  environment  such  as  the  amount  of  care  taken  to  ensure  that  people 
are  informed  of  business  or  mission  goals  and  status,  roles  and  responsibilities,  responsiveness 
to  staff  inputs,  and  the  attitude  and  morale  of  operational  personnel.  The  following  attributes 
characterize  the  Work  Environment  element. 

a.  Quality  Attitude 

The  Quality  Attitude  attribute  describes  the  tendency  of  operational  personnel  to  do 
quality  work  in  general  and  to  conform  to  specific  quality  standards  for  the  mission  and 
the  resulting  products  or  services.  It  involves  recognition  and  reward  of  positive  quality 
attitudes  and  behaviors. 

b.  Cooperation 

The  Cooperation  attribute  is  demonstrated  by  the  level  of  teamwork  among  the  opera¬ 
tional  personnel  both  within  and  across  work  groups.  Managers  at  all  levels  support 
cooperation  by  removing  barriers,  establishing  a  common  vision,  and  negotiating  mutu¬ 
ally  acceptable  goals.  Managers  should  encourage  strong  working  relationships  across 
the  operational  organization. 

c.  Communication 

Good  communication  is  essential  to  establishing  an  operational  environment  in  which 
cooperation  is  strong,  morale  is  good,  and  there  is  pride  in  the  quality  of  the  results. 
Communicating  information  about  the  importance  of  the  mission,  the  status  of  the  mis¬ 
sion  outputs,  and  feedback  from  consumers  of  the  products  and  services  affects  all 
aspects  of  the  work  environment. 

d.  Morale 

Risks  resulting  from  low  morale  can  include  low  enthusiasm  resulting  in  low  perfor¬ 
mance  or  productivity;  anger  that  can  result  in  intentional  damage  to  the  mission;  mass 
exodus  from  the  operational  organization;  and  earning  a  reputation  that  makes  it  diffi¬ 
cult  to  recruit  personnel  within  the  organization. 
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C.  Constraints 


The  Constraints  class  contains  elements  and  attributes  that  are  used  to  characterize  the  external 
factors  that  present  special  challenges  for  operational  organization.  Constraints  can  include 
obtaining  adequate  resources,  compliance  with  regard  to  legal  factors,  or  handling  conflicting 
political  influences. 

1.  Resources 

The  Resources  element  contains  attributes  that  are  used  to  describe  the  resources  on  which  the 
mission  is  dependent  and  factors  outside  the  operational  organization’s  purview  that  are  its 
responsibility  to  obtain  and  maintain.  The  following  attributes  characterize  the  Resources  ele¬ 
ment. 


a.  Schedule 

The  Schedule  attribute  refers  to  the  timeline  and  throughput  of  the  mission  that  is 
required  for  achieving  its  objectives.  Risks  can  be  associated  with  unrealistic  schedule 
constraints  or  overall  operational  throughput.  Schedules  for  installations  or  upgrades  of 
systems  in  addition  to  external  constraints  and  expectations  can  also  cause  risks  to  the 
mission. 

b.  Staff 

The  Staff  attribute  describes  the  stability  and  adequacy  of  the  staff  in  terms  of  numbers 
and  skill  levels,  their  experience  and  skills  in  the  required  technical  areas,  and  their 
availability  when  needed.  Inadequate  staffing  can  result  in  an  inability  to  meet  mission 
requirements,  mission  timeliness,  quality  requirements,  or  operational  infrastructure 
availability. 

c.  Budget 

The  Budget  attribute  describes  the  stability  of  the  budget  with  respect  to  internal  and 
external  events  or  dependencies  and  the  viability  of  plans  and  estimates  for  all  phases 
and  aspects  of  the  maintenance  of  operational  systems. 

d.  Facilities 

The  Facilities  attribute  characterizes  the  adequacy  of  the  operational  facilities  for  plan¬ 
ning  and  executing  the  operational  mission  and  for  communicating  and  for  maintaining 
the  infrastructure  to  support  mission  execution. 
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e.  Tools 

The  Tools  attribute  refers  to  the  availability  of  adequate  tools  to  support  the  manage¬ 
ment,  work,  and  maintenance  processes  of  the  operational  systems.  It  includes  both 
hardware  and  software  tools. 

2.  Policies 

The  attributes  that  compose  the  Policy  element  are  used  to  characterize  risks  associated  with 
laws  and  regulations,  restrictions,  and  contractual  constraints  that  may  affect  the  ability  of  the 
operational  organization  to  perform  its  mission.  The  following  attributes  characterize  the  Pol¬ 
icy  element. 

a.  Laws  and  Regulations 

The  Laws  and  Regulations  attribute  addresses  federal,  state,  local,  or  international  laws 
and  regulations  that  can  impact  the  mission.  Personnel  with  applicable  expertise  may 
not  be  available  to  identify  and  interpret  applicable  laws  and  regulations.  Identifying 
these  risks  is  particularly  important  because  legal  or  regulatory  issues  discovered  late  in 
the  planning  process  can  require  that  the  operational  plans  be  substantially  reworked. 

b.  Restrictions 

The  Restrictions  attribute  describes  constraints  that  can  contribute  risk  to  a  mission. 
Laws,  regulations,  and  local  customs  can  impose  restrictions  to  operational  personnel. 
Local  codes  can  impose  operational  facility  requirements  or  constraints  (e.g.,  electrical 
interfaces,  shock  or  hardness  requirements  for  system  components).  Environmental 
constraints  may  apply. 

c.  Contractual  Constraints 

Contractual  constraints  relate  to  use  of  contracts  by  the  operational  organization.  Con¬ 
tractual  constraints  or  requirements  can  impose  risk  if  the  mission  delivers  products  or 
services  under  contract.  For  example,  risks  can  be  associated  with  incremental  funding 
of  operational  tasks. 

Contractual  constraints  can  also  refer  to  contracts  for  the  physical  space  in  which  the 
operation  takes  place  if  the  space  is  rented  or  leased.  In  this  case,  contractual  constraints 
can  relate  to  property  use  or  configuration. 

Contractual  constraints  can  also  refer  to  labor  contracts  if  the  system  is  operated  by  con¬ 
tract  labor.  For  example,  constraints  that  can  impact  operational  cost  can  exist  for  how 
many  hours  people  can  work,  their  working  conditions,  and  their  benefits. 


CMU/SEI-2005-TR-036 


17 


3.  Interfaces 


The  attributes  that  make  up  the  Interfaces  element  are  used  to  characterize  risks  associated 
with  the  customer  or  user  community,  associate  agencies,  contractors,  senior  leadership,  ven¬ 
dors,  and  politics  external  to  the  operational  organization,  which  can  influence  the  ability  of 
the  operational  organization  to  perform  its  mission.  The  following  attributes  characterize  the 
Interfaces  element. 

a.  Customer/User  Community 

The  Customer/User  Community  attribute  addresses  risks  as  they  relate  to  a  customer’s 
technical  understanding  of  the  mission,  its  products  and  services,  and  the  customer’s 
ability  to  communicate  with  the  operational  organization.  Risks  in  this  category  can 
include  difficult  relationships  or  poorly  conceived  methods  for  requesting  information 
or  services.  Risk  can  also  be  introduced  when  there  are  no  feedback  mechanisms  for 
users  or  customers  to  tell  the  operational  organization  what  improvements  would  bene¬ 
fit  them. 

b.  Associate  Agencies 

The  Associate  Agencies  attribute  describes  risks  with  working  with  other  agencies  or 
organizations.  For  example,  associate  agencies  can  have  conflicting  political  agendas  or 
be  in  competition  for  funding.  Risks  can  be  introduced  in  defining  and  developing  inter¬ 
faces  with  systems  developed  by  other  agencies  or  in  coordinating  schedules  or  config¬ 
uration  changes. 

c.  Contractors 

Using  contractors  to  perform  work  for  an  operational  systems  organization  can  intro¬ 
duce  risk.  Risks  can  arise  from  unclear  or  inadequate  task  definition,  inadequate  or  lack 
of  communication.  If  contractors  are  integral  parts  of  operational  teams,  there  can  be  a 
risk  of  a  lack  of  team  cohesiveness. 

d.  Senior  Leadership 

Senior  leadership  risks  can  include  poor  communication  and  direction  from  senior  man¬ 
agement  and  non-optimal  levels  of  support.  Lack  of  leadership  can  contribute  to  other 
risks  identified  in  the  Work  Environment  element  including  low  morale  and  general 
lack  of  cooperation. 

e.  Vendors 

Vendor  risks  can  be  present  in  dependencies  on  deliveries  and  support  for  maintaining 
operational  capability. 
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f.  Politics 


Political  risks  can  arise  from  relationships  with  associate  agencies  or  other  companies, 
customers,  or  contractors  and  can  affect  technical  decisions  or  ability  to  operate  the  sys¬ 
tem.  Operational  organizations  that  perform  missions  in  international  locations  may 
need  to  spend  extra  care  addressing  issues  that  arise  from  operating  in  foreign  or  remote 
locations. 
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3  Conclusion 


The  taxonomy  for  identifying  and  classifying  risks  to  operational  aspects  of  an  enterprise  pre¬ 
sented  in  this  report  can  help  personnel  identify  the  key  sources  of  risk  associated  with  the 
mission,  work  processes,  and  constraints  of  an  operational  organization.  By  grouping  opera¬ 
tional  risks  into  distinct  classes,  elements,  and  attributes  a  structure  for  representing  opera¬ 
tional  risks  is  established.  In  addition,  personnel  at  operational  sites  can  use  the  short 
taxonomy-based  questionnaire  in  the  appendix  of  this  report  to  identify  and  categorize  risks. 

3.1  Directions  for  Future  Work 

Future  directions  for  this  work  include  developing  methods  to  use  this  taxonomy  to  elicit  oper¬ 
ational  requirements  for  new  system  development,  identify  risks  associated  with  installing 
new  operational  capabilities,  and  conduct  cross-organizational  Team  Risk  Management  activi¬ 
ties  between  operational  personnel,  acquisition  organizations,  and  system  developers.  In  addi¬ 
tion,  the  short  taxonomy-based  questionnaire  found  in  the  appendix  could  be  expanded  to 
allow  for  a  more  in-depth,  structured  exploration  of  operational  risks. 
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Appendix  A  Short  Taxonomy-Based 

Questionnaire  for  Identifying 
Operational  Risks 


A.  Mission 

Consider  risks  to  the  operation  that  can  arise  because  of  the  nature  of  the  mission  that  your 
organization  is  trying  to  accomplish. 

1.  Tasking,  Orders,  and  Plans 

Question:  Are  there  risks  that  could  arise  from  the  way  the  mission  is  tasked,  orders  are  pro¬ 
vided,  or  operational  plans  developed?  Examples: 

a.  Stability  f.  Precedent 

b.  Completeness  g.  Timeliness 

c.  Clarity 

d.  Validity 

e.  Feasibility 

2.  Mission  Execution 

Question:  Are  there  risks  that  could  arise  from  executing  the  mission?  Examples: 

a.  Efficiency 

b.  Effectiveness 

c.  Complexity 

d.  Timeliness 

e.  Safety 
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3.  Product  or  Service 

Question:  Are  there  risks  that  could  arise  from  the  end  product  or  service  this  operational  mis¬ 
sion  provides?  Examples: 

a.  Usability 

b.  Effectiveness 

c.  Timeliness 

d.  Accuracy 

e.  Correctness 


4.  Operational  Systems 

Question:  Are  there  risks  that  could  arise  from  the  operational  systems  used?  Examples: 


a.  Throughput 

b.  Suitability 

c.  Usability 

d.  Familiarity 

e.  Reliability 


f.  Security 

g.  Inventory 

h.  Installations 

i.  System  Support 


Other 

Question:  Are  there  other  risks  that  could  arise  from  your  mission  but  are  not  covered  by  the 
above  categories? 


B.  Work  Processes 

Consider  risks  to  the  mission  that  could  arise  from  the  way  your  organization  is  executing  the 
mission. 
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1.  Operational  Processes 

Question:  Are  there  risks  that  could  arise  from  the  process  the  operational  organization  has 
chosen  to  execute  the  mission?  Examples: 

a.  Formality 

b.  Suitability 

c.  Process  Control 

d.  Familiarity 

e.  Product  Control 

2.  Maintenance  Processes 

Question:  Are  there  risks  that  could  arise  from  the  process  the  front-line  (level  1)  maintenance 
organization  uses  to  maintain  the  operational  systems?  Examples: 

a.  Formality 

b.  Suitability 

c.  Process  Control 

d.  Familiarity 

e.  Service  Quality 

3.  Management  Processes 

Question:  Are  there  risks  that  could  arise  from  the  way  operational  budget  or  schedule  is 
planned,  monitored,  or  controlled;  or  in  the  operational  organization’s  structure;  or  its  handling 
of  internal  and  external  interfaces?  Examples: 

a.  Planning 

b.  Organization 

c.  Management  Experience 

d.  Program  Interfaces 
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4.  Management  Methods 

Question:  Are  there  risks  that  could  arise  from  the  way  operational  personnel  are  managed? 
Examples: 


a.  Monitoring 

b.  Personnel  Management 

c.  Quality  Assurance 

d.  Configuration 
Management 

5.  Work  Environment 

Question:  Are  there  risks  that  could  arise  from  the  general  environment  or  the  larger  organiza¬ 
tion  to  which  the  operational  unit  belongs?  Examples: 

a.  Quality  Attitude 

b.  Cooperation 

c.  Communication 

d.  Morale 

Other 

Question:  Are  there  other  risks  that  could  arise  from  the  way  the  operational  unit  is  going 
about  its  mission  but  are  not  covered  by  the  above  categories? 

C.  Constraints 

Consider  risks  to  the  mission  that  could  arise  from  sources  outside  your  control. 

1 .  Resources 

Question:  Are  there  risks  that  could  arise  from  resources  the  operational  organization  needs 
but  that  are  outside  its  control  to  obtain  or  maintain?  Examples: 

a.  Schedule 

b.  Staff 

c.  Budget 

d.  Facilities 

e.  Tools 
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2.  Policies 


Question:  Are  there  risks  that  could  arise  from  legally  binding  or  constraining  policies? 
Examples: 

a.  Laws  and  Regulations 

b.  Restrictions 

c.  Contractual  Constraints 

3.  interfaces 

Question:  Are  there  risks  that  could  arise  from  outside  interfaces  which  the  operational  orga¬ 
nization  cannot  reasonably  expect  to  control?  Examples: 

a.  Customer/User  f.  Politics 

Community 

b.  Associate  Agencies 

c.  Contractors 

d.  Senior  Leadership 

e.  Vendors 

Other 

Question:  Are  there  other  risks  that  could  arise  from  factors  outside  control  of  the  operational 
organization  but  are  not  covered  by  the  above  categories? 
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